A person recognizes a depth when watching an object because left and right eyes transfer different images to a brain from different positions. A person's brain recognizes a depth of the object due to a phase difference between the two images input through the left and right eyes.
Accordingly, when a stereoscopic image is generated, a pair of images including a left image and a right image needs to be always generated. When a stereoscopic image is generated by a camera, photographing should be performed using two cameras including left and right cameras at the same time. When a video photographed by one camera is converted to a stereoscopic video, two images including left and right images should be provided through one source video. Although both of the methods can make high quality contents, they require a lot of time and efforts.
Recently, a system capable of automatically converting a two-dimensional video to a three-dimensional stereoscopic video without manually corrected by a person has been researched and developed. An image processing algorithm of an automatic stereoscopic video conversion system is divided into various techniques.
First, there is a method using motion parallax. In a sequential frame structure of the video, when a motion in a horizontal direction exists, there is an available method using a current frame and a frame after a proper horizontal motion is detected as left and right images, respectively, as a representative example. This method is commercialized as an internal function of a 3D TV since the method achieves a simple logic and real-time performance.
Second, a method having the same basic idea of using motion information on the video but generating left and right images after recognizing an amount of motion as depth information and generating a depth map or a method of assigning a three-dimensional effect by forwardly disposing a depth value of a foreground after dividing the image into the foreground and a background according to motion information is also actively researched.
However, the methods using the motion information can be used only for making a simple three-dimensional effect, but not in a case when the three-dimensional effect of the video has a slight motion and there is a limitation in making an accurate stereoscopic image due to discontinuity between the motion information and the three-dimensional effect. Since a video having no motion information should newly generate the motion information, a lot of processing time for converting the video to a stereoscopic image is required.
Third, there is a method of generating the three-dimensional effect by using color information without the motion information. The method divides the image into several segmentation areas by using a color value of the image, and then assigns the three-dimensional effect in the unit of segmentations according to a predetermined depth value assignment rule. In general, the assignment of the depth value uses a method of allocating a forward depth value to a bottom area of the image and allocating a backward depth value to a top area of the image. There is also a method of extracting and using only an edge component by using the color information for a real-time processing.
FIGS. 1A to 1D are diagrams illustrating a method of generating a stereoscopic image using color information in the related art.
When pixels having similar color values are grouped into the same area according to color information in an input image as shown in FIG. 1A, the input image is displayed as several division areas as shown in FIG. 1B. Subsequently, when a depth value is backwardly disposed in the unit of division areas from a bottom side to a top side of the image according to a general depth value assignment rule, a depth map as shown in FIG. 1C is generated. FIG. 1D shows a stereoscopic image having left and right parallax generated by shifting an original image according to a depth value of the generated depth map.
In a case of a single image instead of a video, the stereoscopic image can be generated only through the processing procedures such as FIGS. 1A to 1D. However, in a video including consecutive frames, there is no predetermined connectivity between two consecutive frames only with the above procedures, and thus each frame obtains a different division result and, particularly, a depth value is frequently changed without continuity in the depth map sensitive to a position of the division area, so that a flickering phenomenon by which the video is disconnected and a screen is flickering is generated.
Recently, although new technologies for compensating for mutual disadvantages by simultaneously using the motion information and the color information of the two-dimensional video are researched, there are disadvantages in that a processing time is greatly increased while handling further various information.